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Abstract 

The purpose of this study was to investigate the effect of policy at the college level 
(either mandated by the school or optional) of reviewing their instructors through student 
ratings, and the student perceptions of their own ratings. The sample included 954 students 
in "teachers colleges" in the country of Taiwan, during the 1999 spring semester. The 
results indicate some perceptions between "required students"— those required to evaluate, 
and "optional students"— those who chose the afforded option to evaluate, are statistically 
significant. Moreover, they are different, though some of them are not. The significant 
parts include the purposes, concerns, components, and negative effects of the student 
ratings. There is no statistical mean difference on the application of ratings across the two 
groups. Both have generally positive perceptions of the process of student ratings. 
Regardless of whether the school policy mandates the evaluation or not, students agree with 
the following: (1) all instructors should accept the student ratings, (2) the current students 
should be the raters, (3) faculty evaluation committees should be responsible for 
developing the evaluation form, (4) instead of a college-wide evaluation form, a 
department-wide evaluation form should be used for student ratings, (5) the best time to 
implement student ratings is at the end of the semester, (6) the evaluation process may take 
place in the classroom, and (7) the classroom chairman should be the person to take charge 
of the evaluation. 

Key words: Student Ratings, Faculty Evaluation, College Teaching, Teachers College 
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Student Perceptions of Student Ratings: Does School Policy 

Really Matter? 

Introduction 

Student ratings (or called student evaluations) have been officially employed by 
institutions of higher education since 1927. Wagenaar (1995) stated that well over 90 
percent of schools use student ratings for assessing the teaching staff Evidence from many 
other studies also indicates that most universities in North America use student ratings of 
instruction as part of.their evaluation of teaching effectiveness (d'Apollonia & Abrami, 
1997). Seldin (1993) found that student ratings were used as a component of faculty 
evaluation systems by more than 86% of the 600 humanities colleges surveyed. Calderon, 
Gabbin, and Green (1995) found that close to 95 percent of accounting departments use 
student ratings of instruction and as many as 18% rely exclusively on student ratings in 
evaluating faculty teaching performance. As Wilson (1998) predicted, all of the colleges 
would take student ratings of instruction as part of "teaching quality" for their faculty. 

Student ratings of instruction are usually used to provide (1) formative feedback to 
faculty for improving teaching, course content and structure; (2) a summary measure of 
teaching effectiveness for promotion and tenure decisions; and (3) information to students 
for the selection of courses and teachers (Marsh & Roche, 1993). Why are student ratings 
commonly used for faculty evaluations? Some realistic reasons for this are as follows, (1) 
(1) Student raters have closely and recently observed a number of teachers. (2) Students’ 
frank reactions can be a beneficial aid in refining course structure and teaching styles. (3) 
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Student ratings are more objective than other approaches such as administrator evaluations, 
peer evaluation, self-rating, and classroom visitation evaluation. (4) The position of student 
ratings is unique in capacity to indicate how students think and feel. (5) Students are a 
convenient source of rating as well (Arreola, 1995; Peterson, 1995). 

Some researchers have explained that the advent of student consumerism and 
accountability issues in the last two decades has propelled research in student ratings to 
new heights (Benson & Lewis, 1994; Van Patten, 1994). Research on student ratings of 
instruction often examines the issues such as development and validity of an evaluation 
instrument (e.g.. Marsh, 1987), the validity (e.g., Cohen, 1981), and reliability (e.g., 
Feldman, 1977) of student ratings in measuring teaching effectiveness, and the potential 
bias of student ratings (e.g.. Centra & Gaubatz, 2000; Chang, 2000; Feldman, 1993). With 
considerable consistency, research reveals elements of instructor behavior which student’s 
rate to be exceedingly important. With varying labels and weights depending upon the 
individual studies, these elements include the factors of learning value, instructor 
enthusiasm, organization and clarity of explanation, individual rapport, group interaction, 
ability to stimulate student interest and learning, breadth of coverage of material, and 
overall fairness in examinations, grading, assignments, and workload difficulty. 

However, implicit in all of the mentioned literature is the assumption that students 
answer the instruments honestly and willingly. Relatively few studies have attempted to ask 
students either how conscientiously they respond to the questions, or how seriously they 
take the whole evaluation process. In the handful of articles that have addressed the issue, 
the surveyed students have indicated that both the evaluation process is important and that 
students are qualified to rate the their professors. However, students are not optimistic 
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about the overall weight put by administrators and faculty on student opinions (Abbott, 
Wulff, Nyquist, Ropp, & Hess, 1990; Marlin, 1987). 

Like western universities, most colleges and universities in Taiwan have asked for 
feedback from students in rating their quality of the instruction in the classroom. Of all the 
colleges in Taiwan more than 80% have officially employed student ratings of instruction. 
This author has discovered that despite student ratings being one of the most common 
features of faculty evaluation systems, only five of the nine Taiwanese Teacher Colleges 
have mandated policy requiring faculty to implement a student rating in their courses, and, 
the remaining schools offer merely optional student ratings to departments resulting mainly 
from past elements of faculty opposition. Consistently, programs in required-policy 
teacher college faculty are less resisted than those in optional policy faculties. That is, the 
institutional evaluation climate in schools with required policy is higher than schools with 
optional policy. This has become the motivation of this study. As the literature has 
indicated, students are not too optimistic about the overall weight put by administrators and 
faculty on their opinion. Does the mandatory or required policy for student ratings have an 
effect on student perceptions of student ratings? Is there any difference between students in 
required policy (the required students) and students in optional policy (the optional 
students) regarding to perceptions of student ratings? 

Literature Review 

Student Perceptions 

Student ratings of instruction have traditionally served two functions— as formative 
and summative measurements of teaching. One formative use of studied student ratings 
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serves as a source of feedback to instructors who wish to modify their teaching practices. 
Many studies examine the usefulness of student ratings in improving teaching performance 
(Marsh & Roche, 1993; Theall & Franklin, 1991). Student ratings are used to improve 
course content, format, and structure. Studies that examine the course improvement aspect 
of teaching evaluations include Driscoll and Goodwin (1979) and Simpson (1995). 

The summative function of studied student ratings provide information for 
administrations in decisions regarding their faculty. In fact, most colleges and universities 
studied attach great importance to teaching performance in regards to tenure, promotion, 
and pay raise decisions (Cashin & Downey, 1992; Centra, 1994). This summative function 
of teaching evaluation may also provide information for student selection of preferred 
instructors and course sections (Marsh & Roche, 1993). This function though, had been a 
subject of controversy and not yet widely adopted by many colleges and universities. In the 
United States, teaching evaluations have been publicly available information in state- 
supported institutions under the Freedom of Information Act. Studied student groups at 
some universities routinely request this data and disseminate it to the student body (Chen & 
Leon, 1998). 

Since student ratings are used as the primary measure of teaching effectiveness, the 
active participation and meaningful input of students are crucial factors in the success of a 
teaching evaluation system. Important studies have observed a significant linkage between 
student perceptions of student ratings and the success of a teaching evaluation system 
(Douglas & Carroll, 1 987; Marsh, 1 987; Tom, Swanson, & Abbott, 1 990). And few studies 
have made as their primary aim an attempt to look at the wider lens assessment of student 
perspectives on the seriousness of the evaluations, as well as investigate the general 
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satisfaction in the climate of the educational institution. Abbott. Wulff, Nyquist, Ropp, and 
Hess (1990) examine student satisfaction with the evaluation process and emphasize the 
importance of student satisfaction since this may affect a students' willingness to 
participate. Abbott and his colleagues show that students often complain about the 
frequency with which they are asked to fill out rating forms and the degree to which faculty 
are perceived to adjust their courses in response to student feedback. Brandenburg, 
Braskamp, and Ory (1979) are particularly concerned about the possible overuse of student 
rating forms, and question whether students take the evaluations seriously. 

Marlin (1987) found that over half of the students sampled state that they took 
sufficient time and attempted to be fair and accurate in the ratings of their instructors. 
Marlin, however, also states three key findings, (1) students tend to view evaluations as a 
"vent to let off student steam" (2) students often complain that faculty and administrators 
pay scant attention to student opinions, and (3) teacher behavior is not altered based on 
comments from the student rating forms. Jacobs (1987) also reported that 40% of the 
students responding to her survey said that they have heard of students plotting to get back 
at an instructor by collectively giving lower ratings. An overwhelming majority of the 
students reported that they had never heard of an instructor who tried to manipulate 
students into giving higher ratings (Wachtel, 1998). 

Issues for Student Ratings 

To investigate the effects of school policy on student perceptions of student ratings, 
this study focuses on the school policy that provides student ratings as an option or 
requirement for faculty, and the important issues of student ratings. Some key issues of 
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Student ratings have been discussed by various research studies, such student rating 
purposes (e.g., Spencer, 1994), the concerns about ratings (e.g., Abbott, et. al, 1990), the 
components of teaching effectiveness, the negative effect of student ratings (e.g., Wachtel, 
1998), the application of student ratings (e.g. O, 1996), and the procedures of student 
ratings (e.g.. Centra, 1981). 

This study takes the above issues as dependent variables and school policy as the 
independent variable. It is assumed that schools with required policy for student ratings 
have a more mandatory evaluation climate than those with the optional policy. The 
differences in school policies for student ratings might lead to significant differences in 
regard to student perceptions of student ratings. Therefore, it could reasonably be 
speculated that required students and optional students may stand on different positions for 
these important issues of student ratings. 

Conceptual Framework 

The current study takes a structural-functional approach. Colleges possess variable 
climates for the teaching and evaluation activities of the faculty. These different climates 
influence both faculty and students within the context of their teaching and learning 
activities. Student learning and achievement, student growth and satisfaction, student 
development and self-esteem are all potentially important outcomes of the teaching and 
evaluation activities of the schools. Different authors have selected different measures to 
represent these different evaluation activities. And as noted above, student ratings were 
used as the primary measure of teaching effectiveness, and active participation as well as 
meaningful input from students are critical factors in the success of a teaching evaluation 
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system (Chen & Leon, 1998). Besides, Feldman (1987) in his meta-analysis examined the 
many studies using student ratings as a reflection of teaching effectiveness. 

The conceptual framework for this research was derived from the literature on 
academic organizations and indicates the existence of school cultures or climates (Cameron 
& Ettington, 1988). Kolb (1988) claimed that organizational environments can produce 
substantially variable influences on student development. In academic organizations, 
"subunit" effects may be more powerful than the larger organizational culture. Though 
Ewell (1989) found that overall institutional culture is not significantly associated with 
student achievement, and a number of other studies concerned with college impact have 
suggested the importance of sub-environments such the effects of college departments. 
Smart's (1985) study of student self-esteem, for. example, found that the academic sub- 
enviromnents within campuses are more strongly associated with student development than 
are the general campus environments. Similarly, in their study of campus climate, Moran 
and Volkwein (1988) found that subunit effects were more significant than organizational 
effects. Although organizational-level climate did distinguish campuses from one another 
in their study, academic departments accounted for the largest proportion of variance in the 
climate. 

Student perceptions of student ratings result from a variety of campus and student 
individual background influences. Though student individual backgrounds, such as gender, 
GPA, major, year in school, motivation, etc., may contribute to the student perceptions of 
student ratings, these variables are not the focus of the current investigation. This study 
views the school policy as an important component in the student experience and seeks to 
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measure the relationship between organizational evaluation climates and student 
perceptions of students enrolled in those organizations. 

Method 

The Data 

The selected subjects were full time, on campus, undergraduate and graduate 
students from nine Taiwanese Teacher Colleges enrolled during the Spring semester of 
1999. The questionnaire, with a cover letter explaining its purpose, was mailed to a 
systematic sample of 120 students from each college and Table 1 describes the responses 
received. 



Table 1. Biographic Data for Student Sample from Each of Nine Teachers Colleges 



Var 


School 


Schl 


Sch2 


Sch3 


Sch4 


Sch5 


Sch6 


Sch7 


Sch8 


Sch9 


Total 


%of 

group 


School 

policy 


Required 


0 


107 


102 


0 


77 


0 


0 


106 


120 


512 


53.7 


Optional 


103 


0 


0 


105 ; 


0 


120 


114 


0 


0 


442 


46.3 


Gender 


Male 


17 


25 


20 


31 


19 


26 


29 


42 


37 


246 


26.7% 


Female 


63 


80 


81 


74 


58 


97 


79 


56 


87 


675 


73.3% 


Year in 
School 


Freshman 


10 


13 


10 


18 


21 


58 


28 


30 


38 


226 


24,6% 


Sophomore 


19 


48 


33 


38 


20 


40 


66 


24 


30 


318 


34,6% 


Junior 


18 


26 


29 


31 


34 


11 


13 


21 


31 


214 


23,3% 


Senior 


4 


1 1 


30 


10 


2 


6 


2 


11 


20 


96 


10.5% 


Graduate 


20 


5 


0 


8 


0 


10 


0 


15 


6 


. 64 


7.0% 


Total 


N 


103 


107 


102 


105 


77 


120 


114 


106 


120 


954 




% 


10.8% 


11.2% 


10.7% 


11,0% 


8.1% 


12.6% 


11,9% 


11.1% 


12.6% 


100.0% 





iVoie. The number of usable questionnaires is 954. Due to missing data, the valid sample size for the student gender is 922, 
the valid sample size for student year in school is 918. 



An overall response rate was 89.0% {N= 961). Due to missing data, the rate of usable 
questionnaires was further reduced to 88.3% {N= 954) with 512 (53.7%) required students 
and 442 (46.3%) optional students. These 954 student members consisted of 226 (24.6%) 
freshmen, 318 (34.6%) sophomores, 214 (23.3%) juniors, 96 (10.5%) seniors, and 64 
(7.0%) graduate students. 
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The Measurements 

The data was collected by a 42-item survey questionnaire entitled "Questionnaire for 
Student Perceptions of Student Ratings." The questionnaire consisted of four parts. Part I 
contained six demographic items that related to school policy, student gender, academic 
level, GPA, major, and the experience of completing a student rating form. Part II had 33 
items rated on a 5-point Likert-scale ranging from "strongly agree" (5-points) to "strongly 
disagree" (1 -point). It includes five issues (Items 1 - 32) and an overall question item (Item 
33). The five issues concern the purposes of student ratings (Items 1 - 7), the concerns 
about student ratings (Items 8 -1 1), the components of student ratings (Items 12 - 20), the 
negatives of student ratings (Items 21 - 26), and the application of student ratings (Items 27 
- 32). Item 33 asks for an overall agreement of student ratings. 

Part III included a series of procedure questions (Items 34 - 40), such as who should be 
rated, who should be raters, who should be responsible to develop the evaluation form, how 
should the evaluation form be designed, when does the evaluation process take place, and 
where should the evaluation process take place? 

Part IV has two open-ended questions concerning student rating comments about the 
student ratings of faculty teaching (Due to the limitation of the paper, the result of Part IV 
will not be discussed in this paper). 

Analytic Strategy 

The Statistical Package for the Social Science (SPSS) program is utilized to tabulate 
data and compute the statistical tests. Descriptive data is provided to assist with the 
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interpretation of the structure of the relationship between student perceptions of student 
ratings and the school policy. The mean, standard deviation for required-policy and 
optional-policy student responses to each item in Part II of the survey questionnaire is 
accordingly computed and the percentage for each item in Part III is also calculated for each 
group. 

For the purpose of interpreting the pattern of responses to those Likert-scale items 
(Items 1 to 33), the mean of each item for the required students and optional students is 
computed and divided into three levels: disagreement, neutral, and agreement. Thus, a 
mean below 2.49 was treated as a level of disagreement, a mean between 2.50 to 3.49 as a 
level of neutral, and a mean above 3.50 as a level of agreement. 

Since the sample sizes for both required students 512) and optional students 

{Noptionai = 442) are large, a parametric statistic method (independent t test) instead of a 
non-parametric test is applied for testing the mean difference in opinion between the 
required students and optional students on each of the first 33 items. Items in Part III, Items 
34 through 40, were analyzed with the Chi-squared test of homogeneity in order to examine 
the difference in the perceptions of procedure for student ratings between the required 
students and optional students. 

Results and Discussion 

Table 2 shows the mean, standard deviation, and mean differences for the required and 
optional student responses to Items 1 to 33. The majority of the ratings are in the 2.77-4.1 1 
and 2.75-4.14 ranges for respectively, the required and optional students. It appears, 
therefore, that both groups of students stand on agreement or neutral positions for most of 
the items. The first two largest mean differences between the required and optional students 
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are 0.23 with Item 1 1, "Administrators care about the student ratings", and 0.21 with Item 
9, "Professors care about the student ratings". The smallest mean difference is -0.01 with 
Iteml3, "Teaching content and material should be a component of rating". 

Table 2 also presents the results of independent t tests to test the differences in opinion 
between these two group of students. Table 3 presents a ranked order by mean for the items 
of each issue according to policy. The following discussion is based on Table 2 and Table3. 

Purposes of the Ratings 

Among the seven items concerned with purposes of student ratings, both required and 
optional students agree with five of them: "improving teaching", "providing a chance for 
democratic practices of instructors and students", "enhancing the communication between 
instructors and students", "reflecting teaching effectiveness", and "providing a chance for 
self-evaluation on behalf of the instructor." Both required and optional students stand on a 
neutral position for Item 5, "providing reference for faculty promotion" and Item 6, 
"increasing student learning motivation". 

In the required group, the highest mean is 3.97 for Item 7, "providing a chance for 
teacher self-evaluation", while the lowest mean is 3.24 for Item 5, "providing reference for 
faculty promotion." Similarly, in the optional group, the highest mean is 4.08 for Item 7, 
while the lowest mean is 3.27 for Item 5. As shown in Table 3, the required and optional 
students' perceptions of the rank of the seven purposes of student ratings have a perfectly 
positive association = 1.000,/? < .01). 

The mean scores of optional student opinion are significantly higher than those of 
required student opinion on the following items: (1) improving teaching, (2) providing a 
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chance for democratic practices of instructors, (3) enhancing the communication between 
instructor and students, (4) reflecting teaching effectiveness, and (5) providing a chance for 
teacher self-evaluation. It is obvious that the optional students feel more positive on these 
five purposes of student ratings than the required students, although both of them agree 
with the five items. 

Concerns about the Ratings 

Of the four items relating to concerns about the ratings, both required and optional 
students agree with only Item 8, "I care about the student ratings". The required and 
optional students stand on a neutral position for the rest of three items. The lowest means 
are, respectively, 3.06 and 2.83 for Item 1 1, "administrators care about the ratings." In the 
required group, the four items in the ranking order by mean, from highest to lowest, are (1) 
I care about the student ratings, (2) professors care about the student ratings, (3) students 
care about the student ratings, and (4) administrators care about the student ratings. In the 
optional group, the ranks of four items from the highest to lowest are (1) I care about the 
student ratings, (2) students care about the student ratings, (3) professors care about the 
student ratings, and (4) administrators care about the student ratings. 

The mean scores of required student opinions are significantly higher than those of 
optional student opinions on the following two items: (1) professors care about student 
ratings, and (2) administrators care about the student ratings. On the other hand, the 
optional student opinion is significantly higher than those of required students' on (1) I care 
about the student ratings, and (2) students care about the student ratings. Surely students in 
the required policy system feel stronger about their professors and administrators concern 
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over the student ratings but they are less optimistic for student concern and their own. This 
may be due, in part, to a "you-must-do-it" atmosphere created by the required policy itself. 
Some of students may therefore, be simply unwilling to partake in the ratings. Students in 
the optional system feel that their teachers and administrators are not as concerned as they 
should due to optional nature of the evaluation. 

Components of the ratings 

Of the nine items concerned with the components of student ratings, both required and 
optional groups consider the following eight components necessary to the evaluation form: 
personal characteristics of the instructor, teaching content and material, relationship 
between the instructor and students, the teaching skills and method, learning assessment, 
student self-evaluation, instructor overall rating, and the course overall rating. The only 
item which these two groups stand in neutral position over, is Item 18, "student learning 
achievement". In other words, students do not think it is necessary to cover their learning 
achievement in their faculty evaluations. The required students are significantly higher 
than the optional students on this item. Another item, where the optional students are 
significantly higher than the required students, is Item 19, "Instructor overall ratings". 

As shown in Table 3, required and optional students' perceptions of the rank of the 
nine components of student ratings are highly and positively correlated {r^p = 0.971,/? 

< .01). In the required group, the three most important components are (1) teaching skills 
and method, (2.5) instructor overall rating, (2.5) and teaching content and material. In the 
optional group, the three most important components are (1) the overall rating of the 
instructor, (2) teaching skills and method, and (3) teaching content and material. It should 
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be noted in the required group, that the overall rating of the instructor, and teaching content 
and material, tie in the ranking order by their mean. It should also be noted in the ranking 
relationship between instructor, instructor personal characteristics, and student learning 
achievement, the required and optional groups both agree what they consider the three least 
important items. 

The Negative Effect of Ratings 

As for the negative effect of ratings, both required and optional students agree only 
with Item 22, “good teaching may not be rated high,” and further, both groups stand neutral 
for the rest of the items. The fact both groups notice that good teaching may not be rated 
high is confirmed by the finding by Stevens (1978). In a survey of 226 faculty and 572 
students in a southwestern university, Stevens found that faculty and students were 
undecided as to whether student ratings actually measured teacher effectiveness. Both 
studies indicate that the validity of student ratings is still an issue or problem that needs to 
be dealt with. However, it is not only a problem with student ratings, but also a problem for 
all evaluation processes everywhere. Since it is unavoidable to involve personal judgment 
or preferences in the evaluation process, the result of evaluation (including student ratings) 
may be affected by personal values. This might be the reason why faculty, in personal 
discussions, have reservations concerning the use of student ratings and would like to have 
other evaluation systems as well (Jacobs, 1987). 

The mean scores of required student opinions are significantly higher than those of 
optional student opinions on the topic of "reducing teacher energy in teaching" and 
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"following rating results may not be consistent". It seems that required students believe 
more that the student ratings will bring some negative effects on the teaching. 

Required and optional student perceptions on the ranking of the six components of the 
negative effect of student ratings have a perfectly positive association =1 .000,p < .01). 
The two most important components are (1) good teaching may not be indicated or 
reflected in the ratings, and (2) following the rating results may not be consistent. The two 
least effective components according to the two groups are reducing the course 
requirements and influencing the relationship between teacher and the students. 

The Application of the Ratings 

Among six items with application of ratings, both required and optional students agree 
with four items: "for feedback to individual instructor", "for teacher teaching awards", "for 
student course selection", and "for job contract". Both groups stand neutral on "for 
promotion", and "to publish the result in school journal". There is no significant difference 
between these two groups on the six items. 

Required and optional students' perceptions of the rank of the six components of 
application of ratings have a perfectly positive association =1.000,p < .01). The two 
most important components are "for feedback to individual instructor" and "for teacher 



teaching awards". 
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Overall Agreement 

Both required and optional students agree with the implement of written student 
ratings of instruction. Although the mean score of required is 0.03 lower than that of 
optional students, the difference is not statistically significant. 
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Table 2. The Summary of Independent t Test for Student in Required Policy and Student in Optional Policy 
Regarding to Perceptions of Student Ratings (Based on Items 1 to 33) {NraimrcU =512, N,,ptu,nui = 442) 



Item # Issues and item contents 


Required 
M SD 


Optional 
M SD 


Comparison 
M difC t 


The purpose of the ratings 
i To improve the instruction. 


3.85 


0.88 


4.00 


0.73 


•0.15 


-2.780** 


2 


Providing an opportunity for democratic practices on behalf of 3.9! 


0.73 


4.0! 


0.62 


-0.10 


-2.141** 


3 


instructors and students. 

Enhancing the communication between instructor and student 


3.81 


0.84 


3.98 


0.65 


-0.17 


-3.452** 


4 


Reflect teaching effectiveness 


3.70 


0.93 


3.84 


0.80 


-0.14 


-2.321* 


5 


Provide references for faculty promotion 


3.24 


1.02 


3.27 


0.98 


-0.03 


-0.444 


6 


Increasing the student motivation to learn 


3.28 


0.96 


3.32 


0.87 


-0.04 


0.549 


7 


Providing an opportunity for the self-evaluation of the 


3.97 


0.74 


4.08 


0.54 


-0.11 


-2.377* 


instructor. 

The importance of the ratings 
8 I care about the student ratings. 


3.63 


0.92 


3.78 


0.74 


-0.15 


-2.529* 


9 


Professors care about the student ratings. 


3.15 


1.07 


2.94 


l.OI 


0.21 


3.084** 


10 


Students care about the student ratings. 


3.14 


0.98 


3.31 


0.89 


-0.17 


-2.676** 


1 1 


Administrators care about the student ratings. 


3.06 


1.07 


2.83 


1.00 


0.23 


3.214** 


The components of the ratings 

12 The persona! characteristics of the instructor. 


3.77 


0.76 


3.75 


0.76 


0.02 


0.450 


13 


The teaching content and course materials. 


4.09 


0.57 


4.10 


0.45 


-0.01 


-0.062 


14 


The relationship between the instructor and the students. 


3.82 


0.79 


3.77 


0.72 


0.05 


0.983 


15 


Teaching skills and teaching method. 


4.11 


0.58 


4.14 


0.48 


-0.03 


-.0774 


16 


Learning assessment. 


3.97 


0.68 


3.93 


0.58 


0.04 


0.991 


17 


Student self-evaluation. 


3.90 


0.66 


3.86 


0.62 


0.04 


0.961 


18 


Student learning achievements. 


3.32 


0.95 


3.17 


0.87 


0.15 


2.515* 


19 


The overall rating of the instructor. 


4.09 


0.62 


4.16 


0.47 


-0.07 


-2.081* 


20 


The overall rating of the course. 


4.09 


0.62 


4.07 


0.51 


0.02 


-1.539 


The negative effect of ratings 

2 1 Tension for the instructor and the student is a result. 


3.16 


1.02 


3.07 


0.99 


0.09 


1.321 


22 


Good teaching may not be indicated or reflected in the ratings. 


3.84 


0.88 


3.87 


0.86 


-0.03 


-.0509 


23 


Teacher energy is reduced. 


3.03 


1.01 


2.88 


I.OO 


0.15 


2.138* 


24 


Following the rating results may not be consistent. 


3.31 


0.97 


3.13 


1.00 


0.18 


2.710* 


25 


Reduction of course requirements. 


2.77 


0.95 


2.75 


0.93 


0.02 


0.385 


26 


Poorly influences the relationship between teachers and 


2.86 


0.94 


2.77 


0.90 


0.09 


1.473 


students. 

The application and usage of the ratings 

27 To provide a source of feedback to individual instructors. 


3.91 


0.74 


3.96 


0.67 


-0.05 


-I.OIO 


28 


Useful toward the teacher and teaching awards. 


3.70 


0.89 


3.72 


0.83 


-0.02 


-0.366 


29 


For the administrative evaluation of job contracts. 


3.57 


0.93 


3.55 


0.87 


-0.02 


0.306 


30 


For the administrative evaluation of promotions. 


3.35 


0.92 


3.33 


0.90 


-0.02 


-0.283 


31 


For students to select their courses and instructors. 


3.65 


0.88 


3.69 


0.84 


-0.04 


-0.683 


32 


To be published in school journals. 


3.09 


1.07 


3.01 


1.04 


0.08 


1.060 


33 


Overall agreement 


4.28 


0.75 


4.31 


0.67 


-0.03 


-0.734 



Note. < .05; ** p < .01; M diff. = mean difference. 
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Table 3. Ranking Order by Mean for Required and Optional Students' Responses to Items 1-32 {Nreqmrcd- 
512,/V,,,,w = 442) 



Item U Issues and item content 


Required 
M Rank 


Optional 
M Rank 


r.n 


Purpose of the rating 


7 Providing a chance for teacher self-evaluation 


3.97 


1 


4.08 


I 




2 Providing a chance for democracy practice for instructor and 
student 


3.91 


2 


4.01 


2 




1 Improving teaching 


3.85 


3 


4.00 


3 




3 Enhancing the communication between instructor and student 


3.81 


4 


3.98 


4 


1.000** 


4 Reflecting teaching effectiveness 


3.70 


5 


3.84 


5 




6 Increasing student learning motivation 


3.28 


6 


3.32 


6 




5 Providing reference for faculty promotion 


3.24 


7 


3.27 


7 




Importance of ratings 


8 I care about the student ratings 


3.63 


1 


3.78 


1 




9 Professors care about the student ratings 


3.15 


2 


2.94 


3 


0.800 


10 Students care about the student ratings 


3.14 


3 


3.31 


2 




1 1 Administrators care about the student ratings 


3.06 


4 


2.83 


4 




Component of ratings 


15 Teaching skills and method 


4.11 


1 


4.14 


2 




19 Instructor overall rating 


4.09 


2.5 


4.16 


1 




13 Teaching content and material 


4.09 


2.5 


4.10 


3 




20 Course overall rating 


4.09 


4 


4.07 


4 




16 Learning assessment 


3.97 


5 


3.93 


5 


0.97 


1 7 Student self-evaluation 


3.90 


6 


3.86 


6 




14 Relationship between instructor and student 


3.82 


7 


3.77 


7 




12 Instructor personal characteristics 


3.77 


8 


3.75 


8 




18 Student learning achievement 


3.32 


9 


3.17 


9 




Negative effect of ratings 


22 Good-teaching may not be rated high 


3.84 


I 


3.87 


1 




24 Ratings results may not be consistent to follow 


3.31 


2 


3.13 


2 




2 1 Bringing tension to instructor and student 


3.16 


3 


3.07 


3 


1.000** 


23 Reducing teacher energy in teaching 


3.03 


4 


2.88 


4 




26 Influencing the relationship between teacher and student 


2.86 


5, 


2.77 


5 




25 Reducing course requirements 


2.77 


6 


2.75 


6 




Application and usage of ratings 


27 For feedback to individual instructor 


3.91 


I 


3.96 


I 




28 For teacher teaching awards 


3.70 


2 


3.72 


2 




3 1 For student course selection 


3.65 


3 


3.69 


3 


1. 000** 


29 For Job contract 


3.57 


4 


3.55 


4 




30 For promotion 


3.35 


5 


3.33 


5 




32 To publish in school journal 


3.09 


6 


3.01 


6 





Note. ** p < .01 ; r^p — Spearman rank correlation. Instructor overall rating and teaching content and material tied in 
the required group. 
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The process of student ratings 

Who should he rated? As Table 4 indicates, there exists statistically significant 
disagreement in opinion between the required and optional students over this issue. What 
appears to account for the significant difference of opinion on this issue is that the amount 
of “all faculty” selected by optional students (85. 1%) is not nearly as great as it is for 
required students (92.9%). However, more than 85% of the students, no matter what the 
school policy, agree that all faculty in the school should be rated by their students 
concerning their teaching quality. 

Who should be the rater? Item 35 addresses who should be the rater. In the required 
students, 40.3% hold that current students are qualified to be raters, 18.3% hold for 
previous semester students, and 25.8% hold for both current and previous semester 
students. Like the required students, the three relatively high-percentage categories are 
current students (30.1%), "current and previous semester students (27.4%)", and "previous 
semester students (24.3%)" for the optional students. However, there exists statistically 
significant disagreement in opinion between required and optional students over this issue. 
The percentage of required students standing with that current students should be raters is 
much higher than that of optional students. One of the reasons that required students are 
more favorable toward current students is because of the impact on the policy. 

Who should be responsible to develop the evaluation form? As Table 4 indicates, 
there is no statistically significant difference in opinion between the required students and 
optional students on this issue. Faculty evaluation committee is the first choice for the 
required students (37.5%) and optional students (42.9%), student autonomy committee is 
the second choice for the required students (33.6%) and optional students (33.6%). 
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How should the evaluation form be designed? On the issue of the evaluation form 
design, 48. 1% of the required students agree with a department-wide form, 34.3% prefer a 
college-wide form, only 17.7% agree with several options for them to pick one. Similarly, 
most of the optional students (55.6%) agree with a department- wide form and 29.6% agree 
with a college- wide form. There is no statistically difference in opinion between the 
required students and optional students on this issue. 

When does the evaluation process take place? Required students opinion holds fairly 
firm (62.1%) that the end of the semester is the best time to obtain the student ratings. Like 
the required students, the majority of optional students (64.4%) think the student ratings 
process should take place at the end of the semester. It is obvious that both groups regard 
student ratings as summative evaluation rather than formative evaluation. 

Where does the evaluatiooLProcess take place? Item 39 addresses the place where the 
evaluation process takes place. As Table 4 indicates, 49.7 of required students think student 
ratings should be held in the classroom, and 43.3% of optional students feel the same. 
There is no statistically significant difference between the required student’s opinion and 
the optional student’s opinion on this issue. It is noticed that about one fourth of students 
believe that student ratings should be done on the computer network no matter what school 
policy is. 

Who should take charge of the evaluatiomprocess if it is held in the classroom? More 
than 70% of required students hold the opinion that a classroom chairman should be in 
charge of distributing and collecting evaluation forms. Similarly, 72.0% of optional 
students believe that a classroom chairman should be the person to take care of the 
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evaluation process. There is no statistically difference in opinion between the required 
students and optional students on this issue. 



Table 4. The Summary of x' Test for Student in Required Policy and Student in Optional Policy Regarding to 
Perceptions of Student Ratings (Based on Items 34 to 40) = 479, N„pi,„„ai = 402) 



Item # Item Content 


Required N (%) 


Optional N {%) 


7 


34 


Who should be rated? 
All faculty 


445(92.9) 


342(85.1) 


14.508** 




Particular faculty (for promotion) 


22(4.6) 


43(10.7) 






Option to faculty 


12(2.5) 


17(4.2) 




35 


Who should be the raters? 
Current students 


189(40.3) 


124(30.1) 


13.863* 




Previous-semester students 


86(18.3) 


100(24.3) 






Alumni 


7(1.5) 


12(2.9) 






Current and previous-semester students 


121(25.8) 


1 13(27.4) 






Current students and alumni 


5(1.1) 


13(1.9) 






Previous-semester students and alumni 


14(3.0) 


8(3.6) 






All of them 


47(10.0) 


15(9.7) 




36 


Who should be responsible to develop 
the evaluation form? 

Academic affair office 


32(7.0) 


22(5.7) 


3.860 




Department 


100(21.9) 


69(17.8) 






Faculty evaluation committee 


171(37.5) 


166(42.9) 






Student autonomy committee 


153(33.6) 


130(33.6) 




37 


How should the evaluation form be 
designed? 

College-wide 


159(34.3) 


118(29.6) 


4.955 




Department-wide 


223(48.1) 


222(55.6) 






Several options for faculty to pick one 


82(17.7) 


59(14.8) 




38 


When does the evaluation process take 
place? 

The beginning of semester 


15(3.2) 


14(3.5) 


4.089 




The middle of semester 


78(16.6) 


49(12.2) 






The end of semester 


292(62.1) 


259(64.4) 






The beginning and end of semester 


13(2.8) 


15(3.7) 






The middle and end of semester 


65(13.8) 


57(14.2) 






All three sections 


7(1.5) 


8(2.0) 




39 


Where does the evaluation process 
take place? 

In the classroom 


235(49.7) 


177(43.3) 


4.005 




No limitation 


134(28.3) 


124(30.3) 






On the net 


104(22.0) 


108(26.4) 




40 


Who should take charge of the 
evaluation process if it is held in the 
classroom? 

Instructor 


18(3.8) 


23(5.7) 


1.853 




Teaching assistant 


59(12.4) 


48(11.8) 






School officer 


54(11.3) 


43(10.6) 






Class chairman 


346(72.5) 


293(72.0) 





V<-05;*V <-01 
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Conclusion and Recommendations 

The purpose of this study was to ascertain whether there is any relationship between 
school policy and students' perceptions of student ratings. For the first five issues of student 
ratings (as shown in Table 2), some perceptions between the required students and optional 
students are statistically significant different, while some of them are not. The significant 
parts include the purposes, concerns, components, and negative effects of student ratings. 
There is no statistical mean difference on the application of ratings across two groups. The 
findings show that students have strong preferences for the purposes of student ratings and 
these preferences are remarkably consistent across two groups. Nevertheless, the optional 
students agree more the purposes of student ratings in improving teaching, providing a 
chance for democracy practice for instructor and students, enhancing the communication 
between instructor and student, reflecting teaching effectiveness, and providing a chance 
for teacher self-evaluation. 

The required students feel more satisfied than the optional students about that 
professors and administrators care about the student ratings. However, the optional 
students feel more satisfied than the required students about that students care about the 
student ratings. The only item of negative effect of student ratings which both groups agree 
is that good teaching may not be indicated or reflected from the ratings. The rest of items 
for this issue are neutral and not significant different between the required groups and 
optional students. 

Both required students and optional students in teachers colleges have generally 
positive perceptions of the process of student ratings. No matter what kind of school policy 
is, students agree with the following: (1) all instructors should accept the student ratings, 
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(2) the current students should be the raters, (3) faculty evaluation committees should be 
responsible for developing the evaluation form, (4) instead of a college-wide evaluation 
form, a department-wide evaluation form should be used for student ratings, (5) the best 
time to implement student ratings is at the end of the semester, (6) the evaluation process 
may take place in the classroom, and (7) the classroom chairman should be the person to 
take charge of the evaluation. 

Overall, this study provides a baseline for investigating the relationship between 
school policy and student perceptions of student ratings in teachers colleges in Taiwan. It 
suggests that school policy does not affect students’ belief in that students have the right to 
evaluate teacher teaching performance. This increases confidence in the continued use of 
student ratings. However, school policy does make a difference in student perceptions of 
student ratings on their purposes, concerns, negative effect, accuracy, and procedures. This 
gives some warnings for school administrators in implementing student ratings. 

Longitudinal data may provide more details related to the following important 
questions; Why do students in optional policy feel more positively than those of required 
policy on some purposes of student ratings? Why are students in optional policy more 
likely to think that student will express what they really think about the class? Do the 
students in optional policy rate their instructors higher than those in required policy? These 
are student ratings issues for future study. 
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